{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "73b96d3d-406a-405d-850b-38dd7772624e",
   "metadata": {},
   "source": [
    "# 决策树\n",
    "\n",
    "## 目录\n",
    "\n",
    "- 决策树学习器"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72a368ee-02cc-4d92-8d0f-d4323d31d12c",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 决策树学习器\n",
    "\n",
    "### 概述\n",
    "\n",
    "#### 决策树\n",
    "\n",
    "决策树是一个流程图，它使用决策树及其可能的后果进行分类。在树的每个非叶子节点上，输入的一个属性被测试，根据这个测试结果，选择通往子节点的相应分支。在叶子节点上，根据这个叶子节点的类别标签，对输入进行分类。从根到叶的路径代表分类规则，根据这些规则给叶节点分配类标签。\n",
    "\n",
    "![decision tree](decisiontree_fruit.jpg)\n",
    "\n",
    "#### 决策树学习\n",
    "\n",
    "决策树学习是指从有类标记的训练数据中构建决策树。数据预计是一个元组，其中元组的每个记录都是用于分类的属性。决策树是自上而下构建的，通过在每一步选择一个变量来最好地分割项目集。有不同的指标来衡量 \"最佳分割\"。这些指标通常衡量子集内目标变量的同质性。\n",
    "\n",
    "#### 信息增益\n",
    "\n",
    "信息增益是基于信息理论中的熵的概念。熵的定义为：\n",
    "\n",
    "$$H(p) = -\\sum{p_i \\log_2{p_i}}$$\n",
    "\n",
    "信息增益是指父代的熵和子代的熵的加权和之间的差异。用于分割的特征是提供最大信息增益的特征。\n",
    "\n",
    "#### 伪代码"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1744317d-b45d-440c-b3ed-77ab64b6e10e",
   "metadata": {},
   "source": [
    "__function__ DECISION-TREE-LEARNING(_examples_, _attributes_, _parent\\_examples_) __returns__ a tree  \n",
    "&emsp;__if__ _examples_ 是空集 __then return__ PLURALITY\\-VALUE(_parent\\_examples_)  \n",
    "&emsp;__else if__ _examples_ 分类结果都相同 __then return__ 分类结果  \n",
    "&emsp;__else if__ _attributes_ 是空集 __then return__ PLURALITY\\-VALUE(_examples_)  \n",
    "&emsp;__else__  \n",
    "&emsp;&emsp;&emsp;_A_ &larr; argmax<sub>_a_ &isin; _attributes_</sub> IMPORTANCE(_a_, _examples_)  \n",
    "&emsp;&emsp;&emsp;_tree_ &larr; 以特征 _A_ 为根检测节点的决策树  \n",
    "&emsp;&emsp;&emsp;__for each__ 特征 _A_ 的取值 _v<sub>k</sub>_ __do__  \n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;_exs_ &larr; \\{ _e_ : _e_ &isin; _examples_ __and__ _e_._A_ = _v<sub>k</sub>_ \\}  \n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;_subtree_ &larr; DECISION-TREE-LEARNING(_exs_, _attributes_ &minus; _A_, _examples_)  \n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;将标签为 \\(_A_ = _v<sub>k</sub>_\\) 的子树 _subtree_ 添加为 _tree_ 的分支  \n",
    "&emsp;&emsp;&emsp;__return__ _tree_  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8565216-f7c5-4b7f-a791-ebf538a6aaab",
   "metadata": {},
   "source": [
    "### 实现\n",
    "\n",
    "由我们的学习算法构建的树的节点，根据它们是内部节点还是叶节点，分别使用`DecisionFork`或`DecisionLeaf`来存储。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "13283545-a69a-4d96-ab5d-157eaa395f3a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.insert(1, '../')\n",
    "from utils.utils import *\n",
    "\n",
    "from utils.dataset4learners import *\n",
    "from DecisionTreeLearner_3 import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "c3e70aae-3ef5-4e41-b39d-bc0f0b57270a",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\n",
       "   \"http://www.w3.org/TR/html4/strict.dtd\">\n",
       "<!--\n",
       "generated by Pygments <https://pygments.org/>\n",
       "Copyright 2006-2022 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "-->\n",
       "<html>\n",
       "<head>\n",
       "  <title></title>\n",
       "  <meta http-equiv=\"content-type\" content=\"text/html; charset=None\">\n",
       "  <style type=\"text/css\">\n",
       "/*\n",
       "generated by Pygments <https://pygments.org/>\n",
       "Copyright 2006-2022 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "*/\n",
       "pre { line-height: 125%; }\n",
       "td.linenos .normal { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }\n",
       "span.linenos { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }\n",
       "td.linenos .special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }\n",
       "span.linenos.special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }\n",
       "body .hll { background-color: #ffffcc }\n",
       "body { background: #f8f8f8; }\n",
       "body .c { color: #3D7B7B; font-style: italic } /* Comment */\n",
       "body .err { border: 1px solid #FF0000 } /* Error */\n",
       "body .k { color: #008000; font-weight: bold } /* Keyword */\n",
       "body .o { color: #666666 } /* Operator */\n",
       "body .ch { color: #3D7B7B; font-style: italic } /* Comment.Hashbang */\n",
       "body .cm { color: #3D7B7B; font-style: italic } /* Comment.Multiline */\n",
       "body .cp { color: #9C6500 } /* Comment.Preproc */\n",
       "body .cpf { color: #3D7B7B; font-style: italic } /* Comment.PreprocFile */\n",
       "body .c1 { color: #3D7B7B; font-style: italic } /* Comment.Single */\n",
       "body .cs { color: #3D7B7B; font-style: italic } /* Comment.Special */\n",
       "body .gd { color: #A00000 } /* Generic.Deleted */\n",
       "body .ge { font-style: italic } /* Generic.Emph */\n",
       "body .gr { color: #E40000 } /* Generic.Error */\n",
       "body .gh { color: #000080; font-weight: bold } /* Generic.Heading */\n",
       "body .gi { color: #008400 } /* Generic.Inserted */\n",
       "body .go { color: #717171 } /* Generic.Output */\n",
       "body .gp { color: #000080; font-weight: bold } /* Generic.Prompt */\n",
       "body .gs { font-weight: bold } /* Generic.Strong */\n",
       "body .gu { color: #800080; font-weight: bold } /* Generic.Subheading */\n",
       "body .gt { color: #0044DD } /* Generic.Traceback */\n",
       "body .kc { color: #008000; font-weight: bold } /* Keyword.Constant */\n",
       "body .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */\n",
       "body .kn { color: #008000; font-weight: bold } /* Keyword.Namespace */\n",
       "body .kp { color: #008000 } /* Keyword.Pseudo */\n",
       "body .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */\n",
       "body .kt { color: #B00040 } /* Keyword.Type */\n",
       "body .m { color: #666666 } /* Literal.Number */\n",
       "body .s { color: #BA2121 } /* Literal.String */\n",
       "body .na { color: #687822 } /* Name.Attribute */\n",
       "body .nb { color: #008000 } /* Name.Builtin */\n",
       "body .nc { color: #0000FF; font-weight: bold } /* Name.Class */\n",
       "body .no { color: #880000 } /* Name.Constant */\n",
       "body .nd { color: #AA22FF } /* Name.Decorator */\n",
       "body .ni { color: #717171; font-weight: bold } /* Name.Entity */\n",
       "body .ne { color: #CB3F38; font-weight: bold } /* Name.Exception */\n",
       "body .nf { color: #0000FF } /* Name.Function */\n",
       "body .nl { color: #767600 } /* Name.Label */\n",
       "body .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */\n",
       "body .nt { color: #008000; font-weight: bold } /* Name.Tag */\n",
       "body .nv { color: #19177C } /* Name.Variable */\n",
       "body .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */\n",
       "body .w { color: #bbbbbb } /* Text.Whitespace */\n",
       "body .mb { color: #666666 } /* Literal.Number.Bin */\n",
       "body .mf { color: #666666 } /* Literal.Number.Float */\n",
       "body .mh { color: #666666 } /* Literal.Number.Hex */\n",
       "body .mi { color: #666666 } /* Literal.Number.Integer */\n",
       "body .mo { color: #666666 } /* Literal.Number.Oct */\n",
       "body .sa { color: #BA2121 } /* Literal.String.Affix */\n",
       "body .sb { color: #BA2121 } /* Literal.String.Backtick */\n",
       "body .sc { color: #BA2121 } /* Literal.String.Char */\n",
       "body .dl { color: #BA2121 } /* Literal.String.Delimiter */\n",
       "body .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */\n",
       "body .s2 { color: #BA2121 } /* Literal.String.Double */\n",
       "body .se { color: #AA5D1F; font-weight: bold } /* Literal.String.Escape */\n",
       "body .sh { color: #BA2121 } /* Literal.String.Heredoc */\n",
       "body .si { color: #A45A77; font-weight: bold } /* Literal.String.Interpol */\n",
       "body .sx { color: #008000 } /* Literal.String.Other */\n",
       "body .sr { color: #A45A77 } /* Literal.String.Regex */\n",
       "body .s1 { color: #BA2121 } /* Literal.String.Single */\n",
       "body .ss { color: #19177C } /* Literal.String.Symbol */\n",
       "body .bp { color: #008000 } /* Name.Builtin.Pseudo */\n",
       "body .fm { color: #0000FF } /* Name.Function.Magic */\n",
       "body .vc { color: #19177C } /* Name.Variable.Class */\n",
       "body .vg { color: #19177C } /* Name.Variable.Global */\n",
       "body .vi { color: #19177C } /* Name.Variable.Instance */\n",
       "body .vm { color: #19177C } /* Name.Variable.Magic */\n",
       "body .il { color: #666666 } /* Literal.Number.Integer.Long */\n",
       "\n",
       "  </style>\n",
       "</head>\n",
       "<body>\n",
       "<h2></h2>\n",
       "\n",
       "<div class=\"highlight\"><pre><span></span><span class=\"k\">class</span> <span class=\"nc\">DecisionFork</span><span class=\"p\">:</span>\n",
       "<span class=\"w\">    </span><span class=\"sd\">&quot;&quot;&quot;</span>\n",
       "<span class=\"sd\">    A fork of a decision tree holds an attribute to test, and a dict</span>\n",
       "<span class=\"sd\">    of branches, one for each of the attribute&#39;s values.</span>\n",
       "<span class=\"sd\">    &quot;&quot;&quot;</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"fm\">__init__</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">attr</span><span class=\"p\">,</span> <span class=\"n\">attr_name</span><span class=\"o\">=</span><span class=\"kc\">None</span><span class=\"p\">,</span> <span class=\"n\">default_child</span><span class=\"o\">=</span><span class=\"kc\">None</span><span class=\"p\">,</span> <span class=\"n\">branches</span><span class=\"o\">=</span><span class=\"kc\">None</span><span class=\"p\">):</span>\n",
       "<span class=\"w\">        </span><span class=\"sd\">&quot;&quot;&quot;Initialize by saying what attribute this node tests.&quot;&quot;&quot;</span>\n",
       "        <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">attr</span> <span class=\"o\">=</span> <span class=\"n\">attr</span>\n",
       "        <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">attr_name</span> <span class=\"o\">=</span> <span class=\"n\">attr_name</span> <span class=\"ow\">or</span> <span class=\"n\">attr</span>\n",
       "        <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">default_child</span> <span class=\"o\">=</span> <span class=\"n\">default_child</span>\n",
       "        <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">branches</span> <span class=\"o\">=</span> <span class=\"n\">branches</span> <span class=\"ow\">or</span> <span class=\"p\">{}</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"fm\">__call__</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">example</span><span class=\"p\">):</span>\n",
       "<span class=\"w\">        </span><span class=\"sd\">&quot;&quot;&quot;Given an example, classify it using the attribute and the branches.&quot;&quot;&quot;</span>\n",
       "        <span class=\"n\">attr_val</span> <span class=\"o\">=</span> <span class=\"n\">example</span><span class=\"p\">[</span><span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">attr</span><span class=\"p\">]</span>\n",
       "        <span class=\"k\">if</span> <span class=\"n\">attr_val</span> <span class=\"ow\">in</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">branches</span><span class=\"p\">:</span>\n",
       "            <span class=\"k\">return</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">branches</span><span class=\"p\">[</span><span class=\"n\">attr_val</span><span class=\"p\">](</span><span class=\"n\">example</span><span class=\"p\">)</span>\n",
       "        <span class=\"k\">else</span><span class=\"p\">:</span>\n",
       "            <span class=\"c1\"># return default class when attribute is unknown</span>\n",
       "            <span class=\"k\">return</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">default_child</span><span class=\"p\">(</span><span class=\"n\">example</span><span class=\"p\">)</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"nf\">add</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">val</span><span class=\"p\">,</span> <span class=\"n\">subtree</span><span class=\"p\">):</span>\n",
       "<span class=\"w\">        </span><span class=\"sd\">&quot;&quot;&quot;Add a branch. If self.attr = val, go to the given subtree.&quot;&quot;&quot;</span>\n",
       "        <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">branches</span><span class=\"p\">[</span><span class=\"n\">val</span><span class=\"p\">]</span> <span class=\"o\">=</span> <span class=\"n\">subtree</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"nf\">display</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">indent</span><span class=\"o\">=</span><span class=\"mi\">0</span><span class=\"p\">):</span>\n",
       "        <span class=\"n\">name</span> <span class=\"o\">=</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">attr_name</span>\n",
       "        <span class=\"nb\">print</span><span class=\"p\">(</span><span class=\"s1\">&#39;Test&#39;</span><span class=\"p\">,</span> <span class=\"n\">name</span><span class=\"p\">)</span>\n",
       "        <span class=\"k\">for</span> <span class=\"p\">(</span><span class=\"n\">val</span><span class=\"p\">,</span> <span class=\"n\">subtree</span><span class=\"p\">)</span> <span class=\"ow\">in</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">branches</span><span class=\"o\">.</span><span class=\"n\">items</span><span class=\"p\">():</span>\n",
       "            <span class=\"nb\">print</span><span class=\"p\">(</span><span class=\"s1\">&#39; &#39;</span> <span class=\"o\">*</span> <span class=\"mi\">4</span> <span class=\"o\">*</span> <span class=\"n\">indent</span><span class=\"p\">,</span> <span class=\"n\">name</span><span class=\"p\">,</span> <span class=\"s1\">&#39;=&#39;</span><span class=\"p\">,</span> <span class=\"n\">val</span><span class=\"p\">,</span> <span class=\"s1\">&#39;==&gt;&#39;</span><span class=\"p\">,</span> <span class=\"n\">end</span><span class=\"o\">=</span><span class=\"s1\">&#39; &#39;</span><span class=\"p\">)</span>\n",
       "            <span class=\"n\">subtree</span><span class=\"o\">.</span><span class=\"n\">display</span><span class=\"p\">(</span><span class=\"n\">indent</span> <span class=\"o\">+</span> <span class=\"mi\">1</span><span class=\"p\">)</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"fm\">__repr__</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">):</span>\n",
       "        <span class=\"k\">return</span> <span class=\"s1\">&#39;DecisionFork(</span><span class=\"si\">{0!r}</span><span class=\"s1\">, </span><span class=\"si\">{1!r}</span><span class=\"s1\">, </span><span class=\"si\">{2!r}</span><span class=\"s1\">)&#39;</span><span class=\"o\">.</span><span class=\"n\">format</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">attr</span><span class=\"p\">,</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">attr_name</span><span class=\"p\">,</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">branches</span><span class=\"p\">)</span>\n",
       "</pre></div>\n",
       "</body>\n",
       "</html>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "psource(DecisionFork)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ef40288-b753-4155-a813-25153163044f",
   "metadata": {},
   "source": [
    "`DecisionFork`持有属性，在该节点进行测试，以及一个分支的决定。分支存储了子节点，每个属性的值都有一个。以输入元组为参数，以函数形式调用这个类的对象，根据属性测试的结果返回分类路径中的下一个节点。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "79120735-ed3a-4b36-9fb2-5b590de076d3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\n",
       "   \"http://www.w3.org/TR/html4/strict.dtd\">\n",
       "<!--\n",
       "generated by Pygments <https://pygments.org/>\n",
       "Copyright 2006-2022 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "-->\n",
       "<html>\n",
       "<head>\n",
       "  <title></title>\n",
       "  <meta http-equiv=\"content-type\" content=\"text/html; charset=None\">\n",
       "  <style type=\"text/css\">\n",
       "/*\n",
       "generated by Pygments <https://pygments.org/>\n",
       "Copyright 2006-2022 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "*/\n",
       "pre { line-height: 125%; }\n",
       "td.linenos .normal { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }\n",
       "span.linenos { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }\n",
       "td.linenos .special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }\n",
       "span.linenos.special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }\n",
       "body .hll { background-color: #ffffcc }\n",
       "body { background: #f8f8f8; }\n",
       "body .c { color: #3D7B7B; font-style: italic } /* Comment */\n",
       "body .err { border: 1px solid #FF0000 } /* Error */\n",
       "body .k { color: #008000; font-weight: bold } /* Keyword */\n",
       "body .o { color: #666666 } /* Operator */\n",
       "body .ch { color: #3D7B7B; font-style: italic } /* Comment.Hashbang */\n",
       "body .cm { color: #3D7B7B; font-style: italic } /* Comment.Multiline */\n",
       "body .cp { color: #9C6500 } /* Comment.Preproc */\n",
       "body .cpf { color: #3D7B7B; font-style: italic } /* Comment.PreprocFile */\n",
       "body .c1 { color: #3D7B7B; font-style: italic } /* Comment.Single */\n",
       "body .cs { color: #3D7B7B; font-style: italic } /* Comment.Special */\n",
       "body .gd { color: #A00000 } /* Generic.Deleted */\n",
       "body .ge { font-style: italic } /* Generic.Emph */\n",
       "body .gr { color: #E40000 } /* Generic.Error */\n",
       "body .gh { color: #000080; font-weight: bold } /* Generic.Heading */\n",
       "body .gi { color: #008400 } /* Generic.Inserted */\n",
       "body .go { color: #717171 } /* Generic.Output */\n",
       "body .gp { color: #000080; font-weight: bold } /* Generic.Prompt */\n",
       "body .gs { font-weight: bold } /* Generic.Strong */\n",
       "body .gu { color: #800080; font-weight: bold } /* Generic.Subheading */\n",
       "body .gt { color: #0044DD } /* Generic.Traceback */\n",
       "body .kc { color: #008000; font-weight: bold } /* Keyword.Constant */\n",
       "body .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */\n",
       "body .kn { color: #008000; font-weight: bold } /* Keyword.Namespace */\n",
       "body .kp { color: #008000 } /* Keyword.Pseudo */\n",
       "body .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */\n",
       "body .kt { color: #B00040 } /* Keyword.Type */\n",
       "body .m { color: #666666 } /* Literal.Number */\n",
       "body .s { color: #BA2121 } /* Literal.String */\n",
       "body .na { color: #687822 } /* Name.Attribute */\n",
       "body .nb { color: #008000 } /* Name.Builtin */\n",
       "body .nc { color: #0000FF; font-weight: bold } /* Name.Class */\n",
       "body .no { color: #880000 } /* Name.Constant */\n",
       "body .nd { color: #AA22FF } /* Name.Decorator */\n",
       "body .ni { color: #717171; font-weight: bold } /* Name.Entity */\n",
       "body .ne { color: #CB3F38; font-weight: bold } /* Name.Exception */\n",
       "body .nf { color: #0000FF } /* Name.Function */\n",
       "body .nl { color: #767600 } /* Name.Label */\n",
       "body .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */\n",
       "body .nt { color: #008000; font-weight: bold } /* Name.Tag */\n",
       "body .nv { color: #19177C } /* Name.Variable */\n",
       "body .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */\n",
       "body .w { color: #bbbbbb } /* Text.Whitespace */\n",
       "body .mb { color: #666666 } /* Literal.Number.Bin */\n",
       "body .mf { color: #666666 } /* Literal.Number.Float */\n",
       "body .mh { color: #666666 } /* Literal.Number.Hex */\n",
       "body .mi { color: #666666 } /* Literal.Number.Integer */\n",
       "body .mo { color: #666666 } /* Literal.Number.Oct */\n",
       "body .sa { color: #BA2121 } /* Literal.String.Affix */\n",
       "body .sb { color: #BA2121 } /* Literal.String.Backtick */\n",
       "body .sc { color: #BA2121 } /* Literal.String.Char */\n",
       "body .dl { color: #BA2121 } /* Literal.String.Delimiter */\n",
       "body .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */\n",
       "body .s2 { color: #BA2121 } /* Literal.String.Double */\n",
       "body .se { color: #AA5D1F; font-weight: bold } /* Literal.String.Escape */\n",
       "body .sh { color: #BA2121 } /* Literal.String.Heredoc */\n",
       "body .si { color: #A45A77; font-weight: bold } /* Literal.String.Interpol */\n",
       "body .sx { color: #008000 } /* Literal.String.Other */\n",
       "body .sr { color: #A45A77 } /* Literal.String.Regex */\n",
       "body .s1 { color: #BA2121 } /* Literal.String.Single */\n",
       "body .ss { color: #19177C } /* Literal.String.Symbol */\n",
       "body .bp { color: #008000 } /* Name.Builtin.Pseudo */\n",
       "body .fm { color: #0000FF } /* Name.Function.Magic */\n",
       "body .vc { color: #19177C } /* Name.Variable.Class */\n",
       "body .vg { color: #19177C } /* Name.Variable.Global */\n",
       "body .vi { color: #19177C } /* Name.Variable.Instance */\n",
       "body .vm { color: #19177C } /* Name.Variable.Magic */\n",
       "body .il { color: #666666 } /* Literal.Number.Integer.Long */\n",
       "\n",
       "  </style>\n",
       "</head>\n",
       "<body>\n",
       "<h2></h2>\n",
       "\n",
       "<div class=\"highlight\"><pre><span></span><span class=\"k\">class</span> <span class=\"nc\">DecisionLeaf</span><span class=\"p\">:</span>\n",
       "<span class=\"w\">    </span><span class=\"sd\">&quot;&quot;&quot;A leaf of a decision tree holds just a result.&quot;&quot;&quot;</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"fm\">__init__</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">result</span><span class=\"p\">):</span>\n",
       "        <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">result</span> <span class=\"o\">=</span> <span class=\"n\">result</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"fm\">__call__</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">example</span><span class=\"p\">):</span>\n",
       "        <span class=\"k\">return</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">result</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"nf\">display</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">):</span>\n",
       "        <span class=\"nb\">print</span><span class=\"p\">(</span><span class=\"s1\">&#39;RESULT =&#39;</span><span class=\"p\">,</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">result</span><span class=\"p\">)</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"fm\">__repr__</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">):</span>\n",
       "        <span class=\"k\">return</span> <span class=\"nb\">repr</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">result</span><span class=\"p\">)</span>\n",
       "</pre></div>\n",
       "</body>\n",
       "</html>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "psource(DecisionLeaf)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b2935a2-15ff-4fdc-b2b3-930b1539e278",
   "metadata": {},
   "source": [
    "叶子节点在`result`中存储类别标签。所有输入图元的分类路径都在`DecisionLeaf`上结束，其`result` 属性决定了它们的类别。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f272550b-1eac-42fd-b937-29627380b403",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\n",
       "   \"http://www.w3.org/TR/html4/strict.dtd\">\n",
       "<!--\n",
       "generated by Pygments <https://pygments.org/>\n",
       "Copyright 2006-2022 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "-->\n",
       "<html>\n",
       "<head>\n",
       "  <title></title>\n",
       "  <meta http-equiv=\"content-type\" content=\"text/html; charset=None\">\n",
       "  <style type=\"text/css\">\n",
       "/*\n",
       "generated by Pygments <https://pygments.org/>\n",
       "Copyright 2006-2022 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "*/\n",
       "pre { line-height: 125%; }\n",
       "td.linenos .normal { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }\n",
       "span.linenos { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }\n",
       "td.linenos .special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }\n",
       "span.linenos.special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }\n",
       "body .hll { background-color: #ffffcc }\n",
       "body { background: #f8f8f8; }\n",
       "body .c { color: #3D7B7B; font-style: italic } /* Comment */\n",
       "body .err { border: 1px solid #FF0000 } /* Error */\n",
       "body .k { color: #008000; font-weight: bold } /* Keyword */\n",
       "body .o { color: #666666 } /* Operator */\n",
       "body .ch { color: #3D7B7B; font-style: italic } /* Comment.Hashbang */\n",
       "body .cm { color: #3D7B7B; font-style: italic } /* Comment.Multiline */\n",
       "body .cp { color: #9C6500 } /* Comment.Preproc */\n",
       "body .cpf { color: #3D7B7B; font-style: italic } /* Comment.PreprocFile */\n",
       "body .c1 { color: #3D7B7B; font-style: italic } /* Comment.Single */\n",
       "body .cs { color: #3D7B7B; font-style: italic } /* Comment.Special */\n",
       "body .gd { color: #A00000 } /* Generic.Deleted */\n",
       "body .ge { font-style: italic } /* Generic.Emph */\n",
       "body .gr { color: #E40000 } /* Generic.Error */\n",
       "body .gh { color: #000080; font-weight: bold } /* Generic.Heading */\n",
       "body .gi { color: #008400 } /* Generic.Inserted */\n",
       "body .go { color: #717171 } /* Generic.Output */\n",
       "body .gp { color: #000080; font-weight: bold } /* Generic.Prompt */\n",
       "body .gs { font-weight: bold } /* Generic.Strong */\n",
       "body .gu { color: #800080; font-weight: bold } /* Generic.Subheading */\n",
       "body .gt { color: #0044DD } /* Generic.Traceback */\n",
       "body .kc { color: #008000; font-weight: bold } /* Keyword.Constant */\n",
       "body .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */\n",
       "body .kn { color: #008000; font-weight: bold } /* Keyword.Namespace */\n",
       "body .kp { color: #008000 } /* Keyword.Pseudo */\n",
       "body .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */\n",
       "body .kt { color: #B00040 } /* Keyword.Type */\n",
       "body .m { color: #666666 } /* Literal.Number */\n",
       "body .s { color: #BA2121 } /* Literal.String */\n",
       "body .na { color: #687822 } /* Name.Attribute */\n",
       "body .nb { color: #008000 } /* Name.Builtin */\n",
       "body .nc { color: #0000FF; font-weight: bold } /* Name.Class */\n",
       "body .no { color: #880000 } /* Name.Constant */\n",
       "body .nd { color: #AA22FF } /* Name.Decorator */\n",
       "body .ni { color: #717171; font-weight: bold } /* Name.Entity */\n",
       "body .ne { color: #CB3F38; font-weight: bold } /* Name.Exception */\n",
       "body .nf { color: #0000FF } /* Name.Function */\n",
       "body .nl { color: #767600 } /* Name.Label */\n",
       "body .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */\n",
       "body .nt { color: #008000; font-weight: bold } /* Name.Tag */\n",
       "body .nv { color: #19177C } /* Name.Variable */\n",
       "body .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */\n",
       "body .w { color: #bbbbbb } /* Text.Whitespace */\n",
       "body .mb { color: #666666 } /* Literal.Number.Bin */\n",
       "body .mf { color: #666666 } /* Literal.Number.Float */\n",
       "body .mh { color: #666666 } /* Literal.Number.Hex */\n",
       "body .mi { color: #666666 } /* Literal.Number.Integer */\n",
       "body .mo { color: #666666 } /* Literal.Number.Oct */\n",
       "body .sa { color: #BA2121 } /* Literal.String.Affix */\n",
       "body .sb { color: #BA2121 } /* Literal.String.Backtick */\n",
       "body .sc { color: #BA2121 } /* Literal.String.Char */\n",
       "body .dl { color: #BA2121 } /* Literal.String.Delimiter */\n",
       "body .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */\n",
       "body .s2 { color: #BA2121 } /* Literal.String.Double */\n",
       "body .se { color: #AA5D1F; font-weight: bold } /* Literal.String.Escape */\n",
       "body .sh { color: #BA2121 } /* Literal.String.Heredoc */\n",
       "body .si { color: #A45A77; font-weight: bold } /* Literal.String.Interpol */\n",
       "body .sx { color: #008000 } /* Literal.String.Other */\n",
       "body .sr { color: #A45A77 } /* Literal.String.Regex */\n",
       "body .s1 { color: #BA2121 } /* Literal.String.Single */\n",
       "body .ss { color: #19177C } /* Literal.String.Symbol */\n",
       "body .bp { color: #008000 } /* Name.Builtin.Pseudo */\n",
       "body .fm { color: #0000FF } /* Name.Function.Magic */\n",
       "body .vc { color: #19177C } /* Name.Variable.Class */\n",
       "body .vg { color: #19177C } /* Name.Variable.Global */\n",
       "body .vi { color: #19177C } /* Name.Variable.Instance */\n",
       "body .vm { color: #19177C } /* Name.Variable.Magic */\n",
       "body .il { color: #666666 } /* Literal.Number.Integer.Long */\n",
       "\n",
       "  </style>\n",
       "</head>\n",
       "<body>\n",
       "<h2></h2>\n",
       "\n",
       "<div class=\"highlight\"><pre><span></span><span class=\"k\">class</span> <span class=\"nc\">DecisionTreeLearner</span><span class=\"p\">:</span>\n",
       "<span class=\"w\">    </span><span class=\"sd\">&quot;&quot;&quot;DecisionTreeLearner: based on information gain&quot;&quot;&quot;</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"fm\">__init__</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">dataset</span><span class=\"p\">):</span>\n",
       "        <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">dataset</span> <span class=\"o\">=</span> <span class=\"n\">dataset</span>\n",
       "        <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">tree</span> <span class=\"o\">=</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">decision_tree_learning</span><span class=\"p\">(</span><span class=\"n\">dataset</span><span class=\"o\">.</span><span class=\"n\">examples</span><span class=\"p\">,</span> <span class=\"n\">dataset</span><span class=\"o\">.</span><span class=\"n\">inputs</span><span class=\"p\">)</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"nf\">decision_tree_learning</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">examples</span><span class=\"p\">,</span> <span class=\"n\">attrs</span><span class=\"p\">,</span> <span class=\"n\">parent_examples</span><span class=\"o\">=</span><span class=\"p\">()):</span>\n",
       "        <span class=\"k\">raise</span> <span class=\"ne\">NotImplementedError</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"nf\">plurality_value</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">examples</span><span class=\"p\">):</span>\n",
       "<span class=\"w\">        </span><span class=\"sd\">&quot;&quot;&quot;</span>\n",
       "<span class=\"sd\">        Return the most popular target value for this set of examples.</span>\n",
       "<span class=\"sd\">        (If target is binary, this is the majority; otherwise plurality).</span>\n",
       "<span class=\"sd\">        &quot;&quot;&quot;</span>\n",
       "        <span class=\"n\">popular</span> <span class=\"o\">=</span> <span class=\"n\">argmax_random_tie</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">dataset</span><span class=\"o\">.</span><span class=\"n\">values</span><span class=\"p\">[</span><span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">dataset</span><span class=\"o\">.</span><span class=\"n\">target</span><span class=\"p\">],</span>\n",
       "                                    <span class=\"n\">key</span><span class=\"o\">=</span><span class=\"k\">lambda</span> <span class=\"n\">v</span><span class=\"p\">:</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">count</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">dataset</span><span class=\"o\">.</span><span class=\"n\">target</span><span class=\"p\">,</span> <span class=\"n\">v</span><span class=\"p\">,</span> <span class=\"n\">examples</span><span class=\"p\">))</span>\n",
       "        <span class=\"k\">return</span> <span class=\"n\">DecisionLeaf</span><span class=\"p\">(</span><span class=\"n\">popular</span><span class=\"p\">)</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"nf\">count</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">attr</span><span class=\"p\">,</span> <span class=\"n\">val</span><span class=\"p\">,</span> <span class=\"n\">examples</span><span class=\"p\">):</span>\n",
       "<span class=\"w\">        </span><span class=\"sd\">&quot;&quot;&quot;Count the number of examples that have example[attr] = val.&quot;&quot;&quot;</span>\n",
       "        <span class=\"k\">return</span> <span class=\"nb\">sum</span><span class=\"p\">(</span><span class=\"n\">e</span><span class=\"p\">[</span><span class=\"n\">attr</span><span class=\"p\">]</span> <span class=\"o\">==</span> <span class=\"n\">val</span> <span class=\"k\">for</span> <span class=\"n\">e</span> <span class=\"ow\">in</span> <span class=\"n\">examples</span><span class=\"p\">)</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"nf\">all_same_class</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">examples</span><span class=\"p\">):</span>\n",
       "<span class=\"w\">        </span><span class=\"sd\">&quot;&quot;&quot;Are all these examples in the same target class?&quot;&quot;&quot;</span>\n",
       "        <span class=\"n\">class0</span> <span class=\"o\">=</span> <span class=\"n\">examples</span><span class=\"p\">[</span><span class=\"mi\">0</span><span class=\"p\">][</span><span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">dataset</span><span class=\"o\">.</span><span class=\"n\">target</span><span class=\"p\">]</span>\n",
       "        <span class=\"k\">return</span> <span class=\"nb\">all</span><span class=\"p\">(</span><span class=\"n\">e</span><span class=\"p\">[</span><span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">dataset</span><span class=\"o\">.</span><span class=\"n\">target</span><span class=\"p\">]</span> <span class=\"o\">==</span> <span class=\"n\">class0</span> <span class=\"k\">for</span> <span class=\"n\">e</span> <span class=\"ow\">in</span> <span class=\"n\">examples</span><span class=\"p\">)</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"nf\">choose_attribute</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">attrs</span><span class=\"p\">,</span> <span class=\"n\">examples</span><span class=\"p\">):</span>\n",
       "<span class=\"w\">        </span><span class=\"sd\">&quot;&quot;&quot;Choose the attribute with the highest information gain.&quot;&quot;&quot;</span>\n",
       "        <span class=\"k\">return</span> <span class=\"n\">argmax_random_tie</span><span class=\"p\">(</span><span class=\"n\">attrs</span><span class=\"p\">,</span> <span class=\"n\">key</span><span class=\"o\">=</span><span class=\"k\">lambda</span> <span class=\"n\">a</span><span class=\"p\">:</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">information_gain</span><span class=\"p\">(</span><span class=\"n\">a</span><span class=\"p\">,</span> <span class=\"n\">examples</span><span class=\"p\">))</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"nf\">information_gain</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">attr</span><span class=\"p\">,</span> <span class=\"n\">examples</span><span class=\"p\">):</span>\n",
       "<span class=\"w\">        </span><span class=\"sd\">&quot;&quot;&quot;Return the expected reduction in entropy from splitting by attr.&quot;&quot;&quot;</span>\n",
       "        <span class=\"k\">raise</span> <span class=\"ne\">NotImplementedError</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"nf\">split_by</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">attr</span><span class=\"p\">,</span> <span class=\"n\">examples</span><span class=\"p\">):</span>\n",
       "<span class=\"w\">        </span><span class=\"sd\">&quot;&quot;&quot;Return a list of (val, examples) pairs for each val of attr.&quot;&quot;&quot;</span>\n",
       "        <span class=\"k\">return</span> <span class=\"p\">[(</span><span class=\"n\">v</span><span class=\"p\">,</span> <span class=\"p\">[</span><span class=\"n\">e</span> <span class=\"k\">for</span> <span class=\"n\">e</span> <span class=\"ow\">in</span> <span class=\"n\">examples</span> <span class=\"k\">if</span> <span class=\"n\">e</span><span class=\"p\">[</span><span class=\"n\">attr</span><span class=\"p\">]</span> <span class=\"o\">==</span> <span class=\"n\">v</span><span class=\"p\">])</span> <span class=\"k\">for</span> <span class=\"n\">v</span> <span class=\"ow\">in</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">dataset</span><span class=\"o\">.</span><span class=\"n\">values</span><span class=\"p\">[</span><span class=\"n\">attr</span><span class=\"p\">]]</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"nf\">predict</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">x</span><span class=\"p\">):</span>\n",
       "        <span class=\"k\">return</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">tree</span><span class=\"p\">(</span><span class=\"n\">x</span><span class=\"p\">)</span>\n",
       "\n",
       "    <span class=\"k\">def</span> <span class=\"fm\">__call__</span><span class=\"p\">(</span><span class=\"bp\">self</span><span class=\"p\">,</span> <span class=\"n\">x</span><span class=\"p\">):</span>\n",
       "        <span class=\"k\">return</span> <span class=\"bp\">self</span><span class=\"o\">.</span><span class=\"n\">predict</span><span class=\"p\">(</span><span class=\"n\">x</span><span class=\"p\">)</span>\n",
       "</pre></div>\n",
       "</body>\n",
       "</html>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "psource(DecisionTreeLearner)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d22fbe3c-ed37-4c10-83da-170805f822a0",
   "metadata": {},
   "source": [
    "上面的实现使用信息增益作为衡量标准来选择测试哪一个属性进行拆分。该函数以递归的方式自上而下地构建树。根据输入，它做出四个选择中的一个。\n",
    "\n",
    "1. 如果当前步骤的输入没有训练数据，我们将返回在父步骤（上一级递归）中收到的输入数据的类别模式。\n",
    "2. 如果训练数据中的所有值都属于同一类别，它将返回一个`DecisionLeaf`，其类别标签是所有数据所属的类别。\n",
    "3. 如果数据没有可以测试的属性，我们就返回训练数据中具有最高复数值的类。\n",
    "4. 我们选择熵值最高的属性，并返回一个基于此属性的`DecisionFork`。每个分支递归地调用`decision_tree_learning`来构建子树。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b454d72b-421e-48e8-ab0b-aa5fd81e22fb",
   "metadata": {},
   "source": [
    "### 实现要点\n",
    "\n",
    "```py\n",
    "def information_content(values):\n",
    "    \"\"\"Number of bits to represent the probability distribution in values.\"\"\"\n",
    "    probabilities = values 的归一化数值\n",
    "    return probabilities 代入信息熵公式\n",
    "\n",
    "def information_gain(self, attr, examples):\n",
    "    \"\"\"Return the expected reduction in entropy from splitting by attr.\"\"\"\n",
    "\n",
    "    def I(examples):\n",
    "        return information_content([self.count(self.dataset.target, v, examples)\n",
    "                                    for v in self.dataset.values[self.dataset.target]])\n",
    "\n",
    "    n = 样本数\n",
    "    remainder = 剩余信息熵值\n",
    "    return 信息增益\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3227e10-e821-411e-b5b3-8442040b7e47",
   "metadata": {},
   "source": [
    "### 例子\n",
    "\n",
    "现在我们将使用决策树学习器对一个有数值的样本进行分类：5.1, 3.0, 1.1, 0.1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d0e89769-62d8-4057-b2cc-bc165dedefac",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "setosa\n"
     ]
    }
   ],
   "source": [
    "iris = DataSet(name=\"iris\")\n",
    "DTL = DecisionTreeLearner(iris)\n",
    "print(DTL([5.1, 3.0, 1.1, 0.1]))\n",
    "#print(DTL.predict([5.1, 3.0, 1.1, 0.1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c923f2fe-c16c-462c-8d5d-6702094955f4",
   "metadata": {},
   "source": [
    "正如预期的那样，决策树学习器将样本归类为 \"setosa\"。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ffc16d0d-f0c6-4f42-87b3-4f7004ce2c19",
   "metadata": {},
   "outputs": [],
   "source": [
    "assert DTL.predict([5, 3, 1, 0.1]) == 'setosa'\n",
    "assert DTL.predict([6, 5, 3, 1.5]) == 'versicolor'\n",
    "assert DTL.predict([7.5, 4, 6, 2]) == 'virginica'"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
